{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# XGBoost Parameter Tuning for RentListingInquries Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第四步：调整树的参数：subsample 和 colsample_bytree\n",
    "(粗调，参数的步长为0.1；下一步是在粗调最佳参数周围，将步长降为0.05，进行精细调整)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先 import 必要的模块"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "import xgboost as xgb\n",
    "\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from sklearn.metrics import log_loss\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# path to where the data lies\n",
    "#dpath = './data/'\n",
    "train = pd.read_csv(\"RentListingInquries_FE_train.csv\")\n",
    "#train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variable Identification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "选择该数据集是因为的数据特征单一，我们可以在特征工程方面少做些工作，集中精力放在参数调优上"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Target 分布，看看各类样本分布是否均衡"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "每类样本分布不是很均匀，所以交叉验证时也考虑各类样本按比例抽取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_train = train['interest_level']\n",
    "train = train.drop([\"interest_level\"], axis=1)\n",
    "X_train = np.array(train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各类样本不均衡，交叉验证是采用StratifiedKFold，在每折采样时各类样本按比例采样"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# prepare cross validation\n",
    "kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二轮参数调整得到的n_estimators最优值（645），其余参数继续默认值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用交叉验证评价模型性能时，用scoring参数定义评价指标。评价指标是越高越好，因此用一些损失函数当评价指标时，需要再加负号，如neg_log_loss，neg_mean_squared_error 详见sklearn文档：http://scikit-learn.org/stable/modules/model_evaluation.html#log-loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'colsample_bytree': [0.6, 0.7, 0.8, 0.9],\n",
       " 'subsample': [0.3, 0.4, 0.5, 0.6, 0.7, 0.8]}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#max_depth 建议3-10， min_child_weight=1／sqrt(ratio_rare_event) =5.5\n",
    "subsample = [i/10.0 for i in range(3,9)]\n",
    "colsample_bytree = [i/10.0 for i in range(6,10)]\n",
    "param_test3_1 = dict(subsample=subsample, colsample_bytree=colsample_bytree)\n",
    "param_test3_1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/model_selection/_search.py:761: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "([mean: -0.58847, std: 0.00395, params: {'colsample_bytree': 0.6, 'subsample': 0.3},\n",
       "  mean: -0.58718, std: 0.00464, params: {'colsample_bytree': 0.6, 'subsample': 0.4},\n",
       "  mean: -0.58557, std: 0.00367, params: {'colsample_bytree': 0.6, 'subsample': 0.5},\n",
       "  mean: -0.58412, std: 0.00404, params: {'colsample_bytree': 0.6, 'subsample': 0.6},\n",
       "  mean: -0.58490, std: 0.00347, params: {'colsample_bytree': 0.6, 'subsample': 0.7},\n",
       "  mean: -0.58403, std: 0.00368, params: {'colsample_bytree': 0.6, 'subsample': 0.8},\n",
       "  mean: -0.58851, std: 0.00334, params: {'colsample_bytree': 0.7, 'subsample': 0.3},\n",
       "  mean: -0.58600, std: 0.00391, params: {'colsample_bytree': 0.7, 'subsample': 0.4},\n",
       "  mean: -0.58527, std: 0.00385, params: {'colsample_bytree': 0.7, 'subsample': 0.5},\n",
       "  mean: -0.58447, std: 0.00388, params: {'colsample_bytree': 0.7, 'subsample': 0.6},\n",
       "  mean: -0.58424, std: 0.00362, params: {'colsample_bytree': 0.7, 'subsample': 0.7},\n",
       "  mean: -0.58354, std: 0.00435, params: {'colsample_bytree': 0.7, 'subsample': 0.8},\n",
       "  mean: -0.58893, std: 0.00323, params: {'colsample_bytree': 0.8, 'subsample': 0.3},\n",
       "  mean: -0.58603, std: 0.00459, params: {'colsample_bytree': 0.8, 'subsample': 0.4},\n",
       "  mean: -0.58544, std: 0.00427, params: {'colsample_bytree': 0.8, 'subsample': 0.5},\n",
       "  mean: -0.58486, std: 0.00317, params: {'colsample_bytree': 0.8, 'subsample': 0.6},\n",
       "  mean: -0.58419, std: 0.00311, params: {'colsample_bytree': 0.8, 'subsample': 0.7},\n",
       "  mean: -0.58379, std: 0.00316, params: {'colsample_bytree': 0.8, 'subsample': 0.8},\n",
       "  mean: -0.58823, std: 0.00354, params: {'colsample_bytree': 0.9, 'subsample': 0.3},\n",
       "  mean: -0.58588, std: 0.00396, params: {'colsample_bytree': 0.9, 'subsample': 0.4},\n",
       "  mean: -0.58436, std: 0.00401, params: {'colsample_bytree': 0.9, 'subsample': 0.5},\n",
       "  mean: -0.58478, std: 0.00417, params: {'colsample_bytree': 0.9, 'subsample': 0.6},\n",
       "  mean: -0.58397, std: 0.00404, params: {'colsample_bytree': 0.9, 'subsample': 0.7},\n",
       "  mean: -0.58290, std: 0.00331, params: {'colsample_bytree': 0.9, 'subsample': 0.8}],\n",
       " {'colsample_bytree': 0.9, 'subsample': 0.8},\n",
       " -0.5829011882324352)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb3_1 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=251,  #第二轮参数调整得到的n_estimators最优值   ###正在运行...\n",
    "        max_depth=5,\n",
    "        min_child_weight=5,\n",
    "        gamma=0,\n",
    "        subsample=0.3,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "\n",
    "gsearch3_1 = GridSearchCV(xgb3_1, param_grid = param_test3_1, scoring='neg_log_loss',n_jobs=-1, cv=kfold)\n",
    "gsearch3_1.fit(X_train , y_train)\n",
    "\n",
    "gsearch3_1.grid_scores_, gsearch3_1.best_params_,     gsearch3_1.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([ 199.60875211,  214.03226595,  209.93973484,  263.68588905,\n",
       "         216.26374292,  209.82560863,  196.1806334 ,  217.95750732,\n",
       "         225.72279124,  221.29576983,  210.62793713,  198.35448561,\n",
       "         203.09693108,  230.3762969 ,  248.79054618,  253.20724831,\n",
       "         237.49441094,  228.96508479,  224.20137472,  263.31545844,\n",
       "         285.66952667,  295.54662929,  283.29768004,  253.3719151 ]),\n",
       " 'mean_score_time': array([ 0.98750949,  0.843084  ,  0.89697838,  1.49816012,  0.84483895,\n",
       "         0.82098742,  0.78113384,  0.77307181,  0.70904584,  0.6943306 ,\n",
       "         0.6947906 ,  0.69162712,  0.71966615,  0.71079926,  0.70357742,\n",
       "         0.71410127,  0.70391355,  0.69628739,  0.72316036,  0.70783   ,\n",
       "         0.79814291,  0.76081891,  0.72733569,  0.63476434]),\n",
       " 'mean_test_score': array([-0.58847188, -0.58717528, -0.58556705, -0.58411734, -0.58489598,\n",
       "        -0.58403131, -0.58851215, -0.58600394, -0.58527437, -0.58446677,\n",
       "        -0.58423863, -0.5835364 , -0.58892589, -0.5860348 , -0.58544073,\n",
       "        -0.5848584 , -0.58418583, -0.58379415, -0.58823313, -0.58588393,\n",
       "        -0.58435626, -0.58478454, -0.58397447, -0.58290119]),\n",
       " 'mean_train_score': array([-0.51837245, -0.5143835 , -0.51197401, -0.51077645, -0.51071259,\n",
       "        -0.51076741, -0.51548663, -0.51155273, -0.50980544, -0.5080364 ,\n",
       "        -0.50841827, -0.50860002, -0.51413388, -0.51021504, -0.50754471,\n",
       "        -0.50630777, -0.50612972, -0.50681439, -0.51174187, -0.50732442,\n",
       "        -0.50528418, -0.50420885, -0.50457948, -0.50508481]),\n",
       " 'param_colsample_bytree': masked_array(data = [0.6 0.6 0.6 0.6 0.6 0.6 0.7 0.7 0.7 0.7 0.7 0.7 0.8 0.8 0.8 0.8 0.8 0.8\n",
       "  0.9 0.9 0.9 0.9 0.9 0.9],\n",
       "              mask = [False False False False False False False False False False False False\n",
       "  False False False False False False False False False False False False],\n",
       "        fill_value = ?),\n",
       " 'param_subsample': masked_array(data = [0.3 0.4 0.5 0.6 0.7 0.8 0.3 0.4 0.5 0.6 0.7 0.8 0.3 0.4 0.5 0.6 0.7 0.8\n",
       "  0.3 0.4 0.5 0.6 0.7 0.8],\n",
       "              mask = [False False False False False False False False False False False False\n",
       "  False False False False False False False False False False False False],\n",
       "        fill_value = ?),\n",
       " 'params': [{'colsample_bytree': 0.6, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.8}],\n",
       " 'rank_test_score': array([22, 20, 16,  6, 13,  5, 23, 18, 14, 10,  8,  2, 24, 19, 15, 12,  7,\n",
       "         3, 21, 17,  9, 11,  4,  1], dtype=int32),\n",
       " 'split0_test_score': array([-0.58204738, -0.58023044, -0.57986597, -0.57816476, -0.57893572,\n",
       "        -0.57804775, -0.58311   , -0.57936575, -0.58035172, -0.57839853,\n",
       "        -0.57847619, -0.57643722, -0.58374765, -0.57808756, -0.5785873 ,\n",
       "        -0.57947638, -0.57902918, -0.57893316, -0.58403553, -0.57907822,\n",
       "        -0.57700806, -0.57793149, -0.57779461, -0.5779173 ]),\n",
       " 'split0_train_score': array([-0.51830583, -0.51621505, -0.51389584, -0.5118284 , -0.51246856,\n",
       "        -0.51222023, -0.51697156, -0.51283485, -0.51129594, -0.50992552,\n",
       "        -0.5103789 , -0.51009849, -0.5152684 , -0.51123263, -0.50944178,\n",
       "        -0.50768341, -0.50847966, -0.50852664, -0.51281142, -0.50847047,\n",
       "        -0.50685852, -0.50510526, -0.50596253, -0.50546395]),\n",
       " 'split1_test_score': array([-0.58661786, -0.58392051, -0.58357714, -0.58151742, -0.58333865,\n",
       "        -0.58213246, -0.58610937, -0.58409108, -0.5831946 , -0.58194811,\n",
       "        -0.58203402, -0.58137295, -0.5870895 , -0.58448223, -0.5832422 ,\n",
       "        -0.58343852, -0.58216737, -0.58209992, -0.58562958, -0.58436611,\n",
       "        -0.58425441, -0.58342743, -0.58192844, -0.58133369]),\n",
       " 'split1_train_score': array([-0.51815366, -0.51433176, -0.51149036, -0.51000151, -0.51026654,\n",
       "        -0.51109553, -0.5153039 , -0.5127536 , -0.50972223, -0.50808908,\n",
       "        -0.50841894, -0.50805122, -0.51518988, -0.51076132, -0.50728668,\n",
       "        -0.50735947, -0.50642277, -0.50754565, -0.5127107 , -0.50801044,\n",
       "        -0.50496943, -0.50420507, -0.50422836, -0.50628138]),\n",
       " 'split2_test_score': array([-0.58868093, -0.58862697, -0.58557833, -0.58404373, -0.58588999,\n",
       "        -0.5845118 , -0.59122605, -0.5873228 , -0.58367096, -0.58534949,\n",
       "        -0.58513429, -0.58413516, -0.59015986, -0.58664584, -0.5859875 ,\n",
       "        -0.58566   , -0.58618189, -0.58363251, -0.58692294, -0.58672577,\n",
       "        -0.58453751, -0.58439861, -0.58351817, -0.58228502]),\n",
       " 'split2_train_score': array([-0.51784287, -0.51366618, -0.51096402, -0.51103741, -0.51062759,\n",
       "        -0.50947359, -0.51472685, -0.5107559 , -0.50914075, -0.50798151,\n",
       "        -0.50659166, -0.50763235, -0.51376016, -0.50969419, -0.50752008,\n",
       "        -0.50655316, -0.50639473, -0.50625457, -0.51180433, -0.50639744,\n",
       "        -0.50522   , -0.50385947, -0.50402402, -0.50378708]),\n",
       " 'split3_test_score': array([-0.59198059, -0.58951881, -0.58861696, -0.58736317, -0.58808581,\n",
       "        -0.5870819 , -0.5905788 , -0.5889099 , -0.58786295, -0.58745446,\n",
       "        -0.58684916, -0.58697757, -0.59043308, -0.59046839, -0.58850953,\n",
       "        -0.5873484 , -0.58633734, -0.58651817, -0.59097155, -0.58884573,\n",
       "        -0.58787292, -0.58844131, -0.58749994, -0.58564125]),\n",
       " 'split3_train_score': array([-0.51947223, -0.51477922, -0.51162482, -0.51044798, -0.5097795 ,\n",
       "        -0.51051884, -0.51461581, -0.51131035, -0.50955097, -0.50719262,\n",
       "        -0.5088191 , -0.50847939, -0.51352531, -0.51007645, -0.50672117,\n",
       "        -0.50481309, -0.50561722, -0.50647229, -0.51173223, -0.50797806,\n",
       "        -0.50455212, -0.50418266, -0.50543053, -0.50510933]),\n",
       " 'split4_test_score': array([-0.59303403, -0.59358162, -0.59019827, -0.58949926, -0.58823073,\n",
       "        -0.58838395, -0.59153747, -0.59033146, -0.59129344, -0.5891847 ,\n",
       "        -0.58870087, -0.58876066, -0.59320063, -0.59049134, -0.59087878,\n",
       "        -0.58836975, -0.58721427, -0.5877882 , -0.5936077 , -0.59040519,\n",
       "        -0.58810955, -0.58972538, -0.58913275, -0.58733002]),\n",
       " 'split4_train_score': array([-0.51808765, -0.51292529, -0.51189499, -0.51056697, -0.51042077,\n",
       "        -0.51052883, -0.51581504, -0.51010898, -0.50931728, -0.50699329,\n",
       "        -0.50788277, -0.50873867, -0.51292561, -0.50931062, -0.50675382,\n",
       "        -0.50512972, -0.50373422, -0.5052728 , -0.50965068, -0.50576568,\n",
       "        -0.50482084, -0.50369178, -0.50325195, -0.50478231]),\n",
       " 'std_fit_time': array([  3.84796685,  17.63914482,   8.46365978,  21.16515382,\n",
       "          8.40884798,  15.07124901,   5.09196692,   3.60874869,\n",
       "          2.8689317 ,   1.99719158,   1.43247368,   5.5388655 ,\n",
       "          3.78438721,   2.88934773,   2.99386104,   3.75239115,\n",
       "          1.97778648,   1.8530148 ,   2.39437699,   1.8227318 ,\n",
       "          6.86839228,   4.74374699,   4.56732383,  12.57888822]),\n",
       " 'std_score_time': array([ 0.13295881,  0.12565858,  0.14949838,  0.15750715,  0.178575  ,\n",
       "         0.16692314,  0.08123906,  0.09853839,  0.01893062,  0.00218089,\n",
       "         0.01433359,  0.01269325,  0.01456527,  0.01827678,  0.01140295,\n",
       "         0.01048383,  0.01026637,  0.01031332,  0.0121465 ,  0.00756138,\n",
       "         0.09034843,  0.07517665,  0.03920663,  0.11213085]),\n",
       " 'std_test_score': array([ 0.00394659,  0.00463593,  0.00366633,  0.00404215,  0.00347085,\n",
       "         0.00368482,  0.00333904,  0.00391497,  0.00384843,  0.00387632,\n",
       "         0.00362195,  0.00434901,  0.0032327 ,  0.00459261,  0.00426822,\n",
       "         0.00316714,  0.00311288,  0.00315901,  0.00353539,  0.0039634 ,\n",
       "         0.00401233,  0.00416502,  0.00404027,  0.00331046]),\n",
       " 'std_train_score': array([ 0.00056981,  0.00110948,  0.00100753,  0.00062081,  0.00092146,\n",
       "         0.00089566,  0.00085812,  0.00108296,  0.00077115,  0.00103678,\n",
       "         0.00123515,  0.00083866,  0.00093512,  0.00069891,  0.00099698,\n",
       "         0.0011559 ,  0.00152836,  0.00112013,  0.00113663,  0.00104883,\n",
       "         0.00081637,  0.00048861,  0.00098298,  0.00081875])}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gsearch3_1.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.582901 using {'colsample_bytree': 0.9, 'subsample': 0.8}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/wlq666888/APPs/Anaconda3-5.0.1-Linux-x86_64/Anaconda3-5.0.1/lib/python3.6/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3XlcVdUWwPHfZhYHHEBEEGfNCXDA\nAVNTcEorcyp7mUNaZk6lpmaO5cvUzKHUfJWaVpZDpllOYM4DDjiBOaECIqIoCoLAZb0/7hVRUVG4\ngLi/n8/9PO65+5yzz33GYu+zz1pKRNA0TdO0J2WR2x3QNE3Tnm46kGiapmlZogOJpmmaliU6kGia\npmlZogOJpmmaliU6kGiapmlZogOJpmmaliU6kGiapmlZogOJpmmaliVWud2BnODo6CjlypXL7W5o\nmqY9Vfbv339ZRJwe1e6ZCCTlypVj3759ud0NTdO0p4pS6lxm2umpLU3TNC1LdCDRNE3TssSsgUQp\n1UYp9a9S6pRSamQGn/dUSkUrpYJMrz7pPpuilDqmlApRSs1SSinT9teUUodNn00xZ/81TdO0RzPb\nPRKllCXwDdASCAcClVKrRST4nqa/isiAe/b1ARoDHqZN24FmSqkjwFSgrohEK6UWKaV8RcTfXNeh\nadklOTmZ8PBwEhMTc7srmnYXOzs73NzcsLa2fqL9zXmzvT5wSkTOACillgKvAPcGkowIYAfYAAqw\nBqKACsAJEYk2tdsEdAJ0INHyvPDwcAoXLky5cuUwDbA1LdeJCFeuXCE8PJzy5cs/0THMObXlCoSl\nex9u2navTqapquVKqTIAIrIL2AxEml7rRSQEOAU8p5Qqp5SyAjoAZcx4DZqWbRITEylRooQOIlqe\nopSiRIkSWRopmzOQZPRfy73lGNcA5UTEA+PoYhGAUqoSUA1wwxh8WiilmorIVeA94FdgG3AWSMnw\n5Eq9o5Tap5TaFx0dnVETTctxOohoeVFW/12aM5CEc/dowQ24kL6BiFwRkVumt/8D6pp+fhXYLSJx\nIhIH/A00NO2zRkQaiEgj4F/gZEYnF5H5IlJPROo5OT3yeZqM9uf3k7/jf17Pmmmapj2MOQNJIFBZ\nKVVeKWUDvA6sTt9AKeWS7u3LQIjp5/MYb65bKaWsgWa3P1NKlTT9bzGgP/CdOTpvEANL/13KxF0T\nib0Va45TaJqm5QtmCyQikgIMANZjDAK/icgxpdREpdTLpmaDTMt4DwGDgJ6m7cuB08AR4BBwSETW\nmD6bqZQKBnYAk0XkhDn6b2VhxQSfCcTeiuXLfV+a4xSalqOuXbvGnDlznmjfGTNmcPPmzWzry8KF\nCxkwYMCjG2azf/75h/bt2z/2fo/b37Nnz/Lzzz8/9nkyKyYmhpYtW1K5cmVatmzJ1atXM2x3/vx5\nWrVqRbVq1ahevTpnz541S3/M+hyJiPwlIlVEpKKITDJtGysiq00/jxKRGiLiKSLNReS4abtBRN4V\nkWoiUl1EPkx3zG6mbdVFZKk5+/9c8efoWaMnv5/6nd2Ru815Kk0zu7wUSPK7hwWSlJQMb+s+lsmT\nJ+Pr68vJkyfx9fVl8uTJGbZ76623GD58OCEhIezdu5eSJUtm+dwZeSZybWVFP89+bDq/iQk7J7Dy\nlZUUsCqQ213S8oEJa44RfOF6th6zeukijHupxgM/HzlyJKdPn8bLy4uWLVtSsmRJfvvtN27dusWr\nr77KhAkTiI+Pp2vXroSHh2MwGBgzZgxRUVFcuHCB5s2b4+joyObNmzM8/rp16/j4448xGAw4Ojri\n7+9PTEwMvXv35syZM9jb2zN//nw8PDzu2m/ZsmVMmDABS0tLHBwc2Lp1K2fPnqV79+7Ex8cD8PXX\nX+Pj48M///zDuHHjcHZ2JigoiI4dO1KrVi1mzpxJQkICq1atomLFivTs2RM7OzuOHTtGVFQU06dP\nv28kEh8fz8CBAzly5AgpKSmMHz+eV1555YHfX1hYGG3atCE0NJQ33niDcePGMWbMGBwdHRk8eDAA\no0ePxtnZmZ9//pmQkBC8vLzo0aMHxYoVY+3atSQmJhIfH09AQABTp0697/sHWLJkCbNmzSIpKYkG\nDRowZ84cLC0t7+rLH3/8wT///ANAjx49eOGFF/jiiy/uahMcHExKSgotW7YEoFChQg+8tqzSgeQR\n7KzsGNdoHL3X92Zu0Fw+rPfho3fStDxo8uTJHD16lKCgIDZs2MDy5cvZu3cvIsLLL7/M1q1biY6O\npnTp0qxduxaA2NhYHBwcmD59Ops3b8bR0THDY0dHR9O3b1+2bt1K+fLliYmJAWDcuHHUrl2bVatW\nERAQwFtvvUVQUNBd+06cOJH169fj6urKtWvXAChZsiQbN27Ezs6OkydP0q1bt7TEq4cOHSIkJITi\nxYtToUIF+vTpw969e5k5cyazZ89mxowZgHFUsGXLFk6fPk3z5s05derUXeedNGkSLVq04IcffuDa\ntWvUr18fPz8/ChYsmOE17t27l6NHj2Jvb4+3tzft2rXj7bffpmPHjgwePJjU1FSWLl3K3r178fDw\nYNq0afz555+AcWps165dHD58mOLFi7NhwwZOnjx53/fv5OTEr7/+yo4dO7C2tqZ///789NNPvPXW\nW/Tp04d+/fpRr149oqKicHEx3mJ2cXHh0qVL9/X3xIkTFC1alI4dOxIaGoqfnx+TJ0++LyhlBx1I\nMsG7lDedKndiUfAi2pRvQ/US1XO7S9pT7mEjh5ywYcMGNmzYQO3atQGIi4vj5MmTNGnShGHDhjFi\nxAjat29PkyZNMnW83bt307Rp07QH2ooXLw7A9u3bWbFiBQAtWrTgypUrxMbevXilcePG9OzZk65d\nu9KxY0fAmAVgwIABBAUFYWlpyYkTd26Fent7p/0SrVixIq1atQKgVq1ad42WunbtioWFBZUrV6ZC\nhQocP378vu9g9erVTJs2DTA+53P+/HmqVauW4TW2bNmSEiVKANCxY0e2b9/OkCFDKFGiBAcPHiQq\nKoratWuntclo/9vfy4O+/8OHD7N//368vb0BSEhISJuO+u67x1tXlJKSwrZt2zh48CDu7u689tpr\nLFy4kLfffvuxjpMZOpBk0of1PmRr+FbG7RzHz+1+xtriyVIJaFpeICKMGjWKd999977P9u/fz19/\n/cWoUaNo1aoVY8eOzdTxMnoWQeTeR8fuf2Zh3rx57Nmzh7Vr1+Ll5UVQUBCzZ8/G2dmZQ4cOkZqa\nip2dXVp7W1vbtJ8tLCzS3ltYWNx1/+He89z7XkRYsWIFVatWfeT1Pex4ffr0YeHChVy8eJHevXs/\ncP/0I50Hff+zZ8+mR48efP755w/ti7OzM5GRkbi4uBAZGZnhvQ83Nzdq165NhQoVAOjQoQO7d+82\nSyDR2X8zqYhNEUY3GM3xmOP8eOzH3O6Opj22woULc+PGDQBat27NDz/8QFxcHAARERFcunSJCxcu\nYG9vz5tvvsmwYcM4cODAfftmpFGjRmzZsoXQ0FCAtKmtpk2b8tNPPwHGFVOOjo4UKVLkrn1Pnz5N\ngwYNmDhxIo6OjoSFhREbG4uLiwsWFhYsXrwYg8Hw2Ne7bNkyUlNTOX36NGfOnLkvYLRu3ZrZs2en\nBbuDBw8+9HgbN24kJiYm7V5M48aNAXj11VdZt24dgYGBtG7dGnj09/Wg79/X15fly5enTVXFxMRw\n7tz9JUFefvllFi1aBMCiRYsyvLfj7e3N1atXuf1AdkBAANWrm2c2RY9IHoNvWV/83P2Ye2gufmX9\nKFukbG53SdMyrUSJEjRu3JiaNWvStm1b3njjDRo1agQYb8QuWbKEU6dOMXz4cCwsLLC2tmbu3LkA\nvPPOO7Rt2xYXF5cMb7Y7OTkxf/58OnbsSGpqato9jvHjx9OrVy88PDywt7dP++WX3vDhwzl58iQi\ngq+vL56envTv359OnTqxbNkymjdv/sD7Fg9TtWpVmjVrRlRUFPPmzbtrVAMwZswYhgwZgoeHByJC\nuXLl0u5pZOT555+ne/funDp1ijfeeIN69eoBYGNjQ/PmzSlatGja/QcPDw+srKzw9PSkZ8+eFCtW\n7K5jtWrVipCQkPu+/+rVq/PZZ5/RqlUrUlNTsba25ptvvqFs2bJ33SMZOXIkXbt25fvvv8fd3Z1l\ny5YBsG/fPubNm8d3332HpaUl06ZNw9fXFxGhbt269O3b97G/x8xQGQ0985t69epJdlVIvHTzEh1W\ndeC5Es/xfavvdcoLLdNCQkIeOP+uZa+ePXvSvn17OnfubPZzpaamUqdOHZYtW0blypXNfj5zyejf\np1Jqv4jUe9S+emrrMZW0L8nQekMJvBjIypMrc7s7mqblouDgYCpVqoSvr+9THUSySk9tPYGOlTuy\nNnQtX+77kqZuTXGyf/xcXpr2tGrQoAG3bt26a9vixYupVatWLvXofgsXLnyi/davX8+IESPu2la+\nfHl+//33DNtXr16dM2fOPNG58hM9tfUwKUkgBrC+/yHEc9fP0Wl1J5q6NWX6C9OzoZdafqentrS8\nTE9tmYMhGRa+CH8Ny/DjskXK8p7ne2w8txH/czpDsKZpzy4dSB7E0prd1IKDS5BDv2bY5K0ab/Fc\n8eeYtGcS15OyN92Fpmna00IHkgdINqQyy9CJPanPcWvVYK6Fh9zXxtrCmvE+47mSeIWv9n+VC73U\nNE3LfTqQPIC1pQWL+zbmROOvuJlqSdR33dgaHHZfuxolavBW9bdYfmI5gRcDc6GnmqZpuUsHkoew\ntFB0b+3DjTazqEooZ37+kLF/HCUh6e6nbPt79cetkBsTdk0gMeXJ6x5rmjnlpTTyuh5J1mSmHsnm\nzZvx8vJKe9nZ2bFq1Sqz9EcHkkwo26gTKQ3609NqA1F7ltN+9jaOhN9JPFfAqgDjfMZx7vo5vj38\nbS72VNMeLC8FkvwuL9Qjad68OUFBQQQFBREQEIC9vX1agsvspp8jySSrlhMgbDffXP6ezolVeXXO\nTYb4VaZfs4pYWVrQ0KUhHSp1YMHRBbQp14aqxTOXCE57Rv09Ei4eyd5jlqoFbTMucAS6HsmzVo8k\nveXLl9O2bVvs7e0f2CZLRCTfv+rWrSvZ4soZkf+6SfK3LWTgkj1SdsSf0nHODjl3OV5ERK4lXpNm\nS5tJ1zVdJdmQnD3n1PKN4ODgO2/+GiHyw4vZ+/prxEPPHxoaKjVq1BARkfXr10vfvn0lNTVVDAaD\ntGvXTrZs2SLLly+XPn36pO1z7do1EREpW7asREdHP/DYly5dEjc3Nzlz5oyIiFy5ckVERAYMGCDj\nx48XERF/f3/x9PQUEZEFCxbI+++/LyIiNWvWlPDwcBERuXr1qoiIxMfHS0JCgoiInDhxQm7/N7x5\n82ZxcHCQCxcuSGJiopQuXVrGjh0rIiIzZsyQwYMHi4hIjx49pHXr1mIwGOTEiRPi6uoqCQkJsnnz\nZmnXrp2IiIwaNUoWL16cdt7KlStLXFxchte3YMECKVWqlFy+fFlu3rwpNWrUkMDAQAkNDZXatWuL\niIjBYJAKFSrI5cuX7zrP7f1dXV3TvpcHff/BwcHSvn17SUpKEhGR9957TxYtWiQiIm+//bYEBgaK\niIiDg8Nd/StatOgD/78REWnevLmsWbPmoW3u+vdpAuyTTPyO1SOShxAREEFZmGYAi5eHl2ZitbwX\nMxuvxff1d/hk1VHaztzKuJdq0KWeG6MajGLYlmH8FPITPWr0yN0L0PKuh4wccoKuR5L/65HcFhkZ\nyZEjR9IyE5uDDiQPIMnJXBgxApvyFXAamO4mW82OELoVtWMGr7zZhHpDmjL0tyA+WnGYTSFR/PfV\nZrxQ5gW+Pvg1LdxbUKZwmdy7CE17ANH1SPJ9PZLbfvvtN1599VWsrc1XQ0nfbH8QKyuUjS2Xv/mG\nG/fOCbf5HErWgJXv4mpxjZ/7NGT0i9X4599o2szcTvMS/bC0sGTirokZ/oekablB1yN5tuqR3PbL\nL7/QrVu3h15bVulA8gBKKUqNH4dd9epc+GgESWfP3vnQugB0WQDJN2FlXyxIpW/TCvwxoDGOhWz4\n8JezVLR8jd2Ru/nj9B+5dQmadpf09Ug2btyYVo+kVq1adO7cmRs3bnDkyBHq16+Pl5cXkyZN4pNP\nPgHu1CNp3rx5hsdOX4/E09OT1157DYDx48ezb98+PDw8GDly5APrkdSqVYuaNWvStGnTtHokixYt\nomHDhpw4cSJL9Ujatm37wHokycnJeHh4ULNmTcaMGfPQ492uR+Ll5UWnTp3uq0fStWvXDOuRfPXV\n/Q8rt2rVKsPvP309Eg8PD1q2bElkZCRgHPnczhk4cuRINm7cSOXKldm4cSMjR44EjPVI+vTpk3ae\ns2fPEhYWRrNmzR77+3scOmnjIyRHRBDaqTNWTo6UW7oUi/T/oA/+BH/0h+ajodlHACQmG/hyw798\nt/00RSt+h22BaP7suBrHAo7ZcSnaU0wnbcw5uh7J49NJG83I2tWV0l9O49bpM1z45JO7p6q83gCP\n1+Cfz+HsdgDsrC0Z3a46P/VphM3V14hLusnbaz4hxZCaS1egaZq56HokRmYdkSil2gAzAUvgOxGZ\nfM/nPYGpQIRp09ci8p3psylAO4zBbiMwWEREKdUN+BgQ4ALwpohcflg/sqNC4uX//Y/oL6dT8qOP\nKNG7150Pbt2A+S9AUjz02w4F74w8YhOSeWvFJM4YVuCS2J/5nd6inOPjD9G1/CG/jEiehnokT+px\n65HkJ1kZkZgtkCilLIETQEsgHAgEuolIcLo2PYF6IjLgnn19MAaYpqZN24FRpv+9AFQXkcumYHNT\nRMY/rC/ZEUhEhIjBQ7ixaRPuP3xPwYYN73wYeRi+84MKzaDbr2BxZ6CXbEjmxeWduBgXg+H8MMa0\nq8Pr3mV0id5nUH4JJFr+lFentuoDp0TkjIgkAUuBBy8tuJsAdoANYAtYA1GAMr0KKuNv4iIYA4vZ\nKaVw+e9/sSlfnogPPiTZdAMMABcPaD0JTm6A3d/ctZ+1pTXTW0xCWV3Hqaw/o1Yeoe+P+7kcdwtN\n07T8wJyBxBVIny433LTtXp2UUoeVUsuVUmUARGQXsBmINL3Wi0iIiCQD7wFHMI1MgO/NeA13sSxU\nELfZs5GkJMIHDiI1/fDeuw9Uewk2jYfw/XftV8upFv+p9h9iLP+hl6+w9WQ0rb/ayqbgqJzquqZp\nmtmYM5BkNHdz7zzaGqCciHgAm4BFAEqpSkA1wA1j8GmhlGqqlLLGGEhqA6WBwxinvO4/uVLvKKX2\nKaX2RUdHZ8f1AGBboTylv5hM4tGjXJyY7jkRpeDlr6FwaVjeExKu3bXfwNoDKV2wNPvj57PiPW9K\nFrGjz4/7GLXyCPG3sp7ETdM0LbeYM5CEA+kf63bjnmkoEbkiIrf/rP8fUNf086vAbhGJE5E44G+g\nIeBl2u+0KQ/Mb4BPRicXkfkiUk9E6jk5OWXXNQFQ2M+PEv3eJXbFSq79tuzOBwWKQucf4PoFWDMI\n0t1/sre2Z2yjsYTGhrLl0lJWve/Du80qsDTwPO1mbePg+fvTQGtadspL2X91GvmsyUwaeYCPPvqI\nGjVqUK1aNQYNGmS2B6TNGUgCgcpKqfJKKRvgdWB1+gZKKZd0b18GbpchPA80U0pZmUYhzUyfRQDV\nlVK3I0PLdPvkKKeBAyn4/PNc/OwzEoKC7nxQxht8x0LwH7Dvh7v2aezamJcqvMT3R77n3I3TjGpb\njV/6NiTZIHSet4uvNp4gWS8T1swkLwWS/C4vpJHfuXMnO3bs4PDhwxw9epTAwEC2bNmS5XNnxGyB\nRERSgAHAeoy/7H8TkWNKqYlKqZdNzQYppY4ppQ4Bg4Cepu3LgdMY74UcAg6JyBoRuQBMALYqpQ5j\nHKH811zX8DDK0hLXaVOxdnYmfPAQUi6nW4HcaCBUagnrRt2XKny493AK2xRmws4JGFINNKxQgr+H\nNOEVz9LM9D9J57k7ORMdl8NXoz0L0qeRHz58OFOnTsXb2xsPDw/GjRsHGFOrt2vXDk9PT2rWrMmv\nv/7KrFmz0tLIP+jJdjCmka9Tpw6enp74+voCxr+cO3TogIeHBw0bNuTw4cP37bds2TJq1qyJp6cn\nTZsaF2qePXuWJk2aUKdOHerUqcPOnTsB44iiWbNmdO3alSpVqjBy5Eh++ukn6tevT61atTh9+jRg\nfCCxX79+NGnShCpVqvDnn3/ed974+Hh69+6Nt7c3tWvX5o8/Hp6F4nYa+apVq6alfB8zZgwzZ85M\nazN69GhmzZrFyJEj2bZtG15eXnz11VcsXLiQLl268NJLL6Ulmczo+wdjGvnb2QXefffdDNPD/PHH\nH/ToYUwK26NHjwwLVimlSExMJCkpiVu3bpGcnIyzs/NDr/GJZSZF8NP+yrY08hlICA6WEE8vOftm\nd0k1pX4WEZG4aJFpVUVm1RVJvHHXPmtPr5WaC2vK4mOL79r+56EL4jF+vTz3yd+yeNdZSU1NNVu/\ntZyXPk335D2TpeffPbP1NXnP5IeeX6eRf/bSyA8dOlQcHBykSJEi8vHHH2fY5raspJHXT7ZnkV21\narhMnMDNwEAumdJRA8YHEzv+D2JOw1/D79qnbfm2NHFtwqyDs4iIi0jb3s7DhfVDmlKvXDE+WXWU\ntxft49INXbpXy37p05jXqVOH48ePc/LkSWrVqsWmTZsYMWIE27Ztw8HBIVPHe1ga+e7duwOPTiP/\nv//9L+2v7+TkZPr27UutWrXo0qULwcFpj5+lpZG3tbW9L4382XQ58TKTRn7y5Ml4eXnxwgsvpKWR\nf5DbaeQLFCiQlka+XLlyaWnkb3+fj5tGPv337+/vn5ZG3svLC39/f86cOQMY08jfzu+VGadOnSIk\nJITw8HAiIiIICAhg69atmd7/ceg08tnA4eWXSThylJhFP2JXsxYOL5lu5pVvAk0/gi2ToXxT8DJm\n4FRKMabhGF754xU+3fUpc/3mpj2gWMrBjkW96rNo11km/32cNjO2MbljLVrVKJVLV6eZw4j6Ix7d\nyIxEp5HP92nkf//9dxo2bEihQoUAaNu2bVrAz256RJJNnD8aToF6dYkcM4bEf/+980Gzj6BcE1g7\nFC6fTNvsUsiFwXUGs+PCDv48c/f8rYWFolfj8vw58HlcHOx4Z/F+Riw/TJxeJqxlgU4j/2ylkXd3\nd2fLli2kpKSQnJzMli1bzJZZQQeSbKKsrXH76issCxcmfMBADLeH7xaWxikuaztY1hOSE9L2eb3q\n63g6eTIlcAoxiTH3HbOyc2F+79+Y/i9U5Lf9Ybw4cxv7z+llwtqT0Wnkn6008p07d6ZixYrUqlUL\nT09PPD09eemllx77e8yUzNxIedpf5rzZfq/4AwckuGYtOde3r6QaDHc+OLFBZFwRkTUf3NX+ZMxJ\n8frRS0ZsfXi97b2hV6TxZH8pP/JPmbb+uCSlGB7aXst7MrqZqZlHjx49ZNmyZTlyLoPBIJ6ennLi\nxIkcOZ+56JvteYh97dqUGv0x8Vu3cfnrdHm3KrcEn0Gw73s4dmepXqVilehbqy9rz6xlW/i2Bx7X\nu1xx/h7chI513JgdcIqOc3Zy6pJeJqxpuUmnkTfSha3MQESIHP0JsStX4jbnGwq3aGH8wJAMP7SB\nyyeg3zYoVg6AJEMSXdZ04WbKTVa9soqC1g8fxq87GsmolUdISDbw8YvV6N6wrM4m/BTIL9l/dRr5\n/ClPppHPS3I6kACkJiZy7j9vknTuHOWW/YataVkkV8/BvCbgWAl6rQMrGwCCLgXx1t9v8Ua1NxhZ\nf+Qjj3/peiLDlx9my4lomlVxYmpnD0oWsXvkflruyS+BRMuf8moa+WeahZ0dbrNmoqysCB84kNT4\neOMHxcrCK7MhYj8ETExr71XSi9efe52fQ34m6FLQA456R8kidizs5c3EV2qwJ/QKrWdsZd3RyEfu\np2malt10IDEja1dXXKd/SdKZUC6MTlemt/orxrTzO2fDifVp7QfXGYxzQWfG7xxPsiH5kcdXSvFW\no3L8ObAJbsXs6bfkAMOWHeJG4qP31TRNyy46kJhZQR8fSn74ATfWrSPmhwV3Pmg1CZxrwe/9jNmC\ngYLWBRnTcAynY0/z3dHvMn2OSiULsbK/DwNbVGLlgXDaztxG4Nn7lxNrmqaZgw4kOaD4229TuHVr\nLn35JfG7dxs3WttBlwWQcgtW9AGD8WHDpm5NaVu+LfMPz+f0tdOZPoe1pQVDW1VlWb9GWCjFa9/u\nYsq64ySl6GzCmqaZlw4kOUAphcukSdhUMJXpvWAqy+JYGdpPh3M7YOuUtPYjvEdQ0Log43eOJ1Ue\nLxDULVucvwY3oUvdMsz55zSvztnByagHP2GrPTvyUhp5XY8kazJbj2TEiBHUrFkzLZOzuehAkkMs\nCxXEbdZsJDn57jK9nq+D139gyxQINSZUK1GgBB95f0RQdBC//vv4/+cXsrXii84efNu9LpGxibSf\nvZ2FO0JJTc3/K/S0B8tLgSS/ywv1SNauXcuBAwcICgpiz549TJ06levXr2f53BnRgSQHpZXpPXaM\nixPSlel9capxdLKiL8QZywK/VOElfEr7MGP/DC7GX3yi87WuUYp1Q5rgU7EE49cE02PBXi7G6mzC\nzypdj+Ru+b0eSXBwMM2aNcPKyoqCBQvi6enJunXrHnqNTywzj78/7a+cTJGSGVEzZkhw1eck5pel\ndzZGHhGZ6CTy46siptQqYdfDxHuJt/Tf1D9LtUlSU1Nl8a6z8twnf4vH+PXy56ELWb0E7QmkT0ER\nOWmSnH2ze7a+IidNeuj5dT2SZ6seyfr168XHx0fi4+MlOjpaypcvL9OmTXvQ/4VZSpGi08jnAqcB\nA4yjkkmTsHuuKgW8vKBUTWg7Gf78AHbOgueH4FbYjQFeA5i6byrrzq6jbfm2T3Q+pRRvNiyLT8US\nfPBrEO//fAD/EFfGv1KDInbW2Xx12tMgfT0MgLi4OE6ePEmTJk0YNmwYI0aMoH379jRp0iRTx3tY\nPZIVK1YAj65H0rVrVzp27AgY65EMGDCAoKAgLC0tOXHiRFr72/VIgPvqkWzevDmtXWbqkaxevZpp\npjpCt+uRPOih0dv1SIC0eiRTV2ePAAAgAElEQVRDhgxJq0cSFRX1RPVI4M73f/jw4bR6JAAJCQlp\nKeK/+y7zKznBmBgyMDAQHx8fnJycaNSoEVZW5vmVrwNJLlCWlrhOmUJol66EDxpM+ZUrsHJ0hLq9\n4MwWCPgUyvpAmfr8p9p/+Dv0bybvnUwjl0YUtSv6xOet4FSI5e/5MDvgFN9sPsWe0Bimd/WkQYWM\n/+Fr5lPq449z9fwiuh6JSP6uRwLGqbbRo0cD8MYbb5gtH5i+R5JLLIsWxW32LAzXrxMx5AMkORmU\ngpdnQRFXWN4bEq5iaWHJeJ/xXL91nan7pmb5vNaWFnzYsgrL+jXCylLx+v928/nfIdxKefx6D9rT\nRdcjebbqkRgMBq5cuQLA4cOHOXz4cNroLbvpQJKL7J57DpdPJ3Jz3z6ippqChJ2D8fmSGxfhjwEg\nQtXiVelVsxerT69mZ8TObDl3Hfdi/DWoCa97u/PtljN0+GYnJ/Qy4XxN1yN5tuqRJCcn06RJE6pX\nr84777zDkiVLzDa1les3wnPilddutt8rctIkCa76nFxbvfrOxh2zjfVLdn8rIiKJKYnSfmV7ab28\ntcQnxWfr+Tceuyh1Jm6QyqP/ku+2nRGD4clv7GsPpuuR5Bxdj+Tx6XokTznn4cOxr1ePyDFjSbx9\nQ7DR+1C5NWwYDZGHsLW0ZbzPeCLiIvgm6JuHH/Ax+VV3Zv0HTWla2ZFP/wzm3SX7uZmky/pq2qPo\neiRGOo18HpFy+TKhHTuhbGwov3wZlkWLQvwVmPc8WBeAd7eAbWE+3fUpy08u56cXf6KmY81s7YOI\nsGDHWT5bG0xNVwe+61GPkoV1avrskl/SyOt6JPmTrkfyCE9DIAFICAribPe3KNiwIWXmzUVZWsK5\nnbCwHdTsDB3ncyM5jg6rOuBg58Cv7X/F2iL7l+9uDI5i0C8HKV7QhoW9vKnsXDjbz/Esyi+BRMuf\n8mw9EqVUG6XUv0qpU0qp+6o1KaV6KqWilVJBplefdJ9NUUodU0qFKKVmKaPC6doGKaUuK6VmmPMa\nclIBLy9KjR5N/LZtXP7GNH1V1gdeGAVHfoOgnyhsU5jRDUdz8upJFh5daJZ+tKzuzK/vNuRWSiod\n5+5k5+nLZjnPs+hZ+MNNe/pk9d+l2QKJUsoS+AZoC1QHuimlqmfQ9FcR8TK9vjPt6wM0BjyAmoA3\n0ExEbqRr6wWcA1aa6xpyQ9HXuuLQqSOX58zlhr+/cWOToVC+Kfw1HC4dp4V7C1qVbcW8Q/MIjQ01\nSz883Iqy6n0fShWxo8cPe1mxP9ws53mW2NnZceXKFR1MtDxFRLhy5cp9q9oeh9mmtpRSjYDxItLa\n9H4UgIh8nq5NT6CeiAzIYN+vgecBBWwFuotISLo2lYEAwF0ecRFPy9TWbam3bhnL9IaGUm7ZMmwr\nlDcuB57bGAqVhL4BXE6J5+VVL1O5aGUWtFmAhTLP3wSxCcm8t2Q/O09fYYhfZQb7Vtb14Z9QcnIy\n4eHhJCbqfGda3mJnZ4ebmxvW1ndPlWd2asucT7a7AmHp3ocDDTJo10kp1RQ4AXwgImEisksptRmI\nxBhIvk4fREy6YRzNZBhElFLvAO8AuLu7Z+1KcpiFrS1us2YS2qkz4YMGUm7pr1gWLgUdv4UlnWDd\nSBxfmsnwesMZu3Msy08sp2vVrmbpi0MBaxb2qs+olUeYsekk52NuMrmjBzZWesHf47K2tk5LIaJp\n+Yk5fxtk9Gfrvb/01wDlRMQD2AQsAlBKVQKqAW4YA1ILU7BJ73XglwedXETmi0g9Eann5OT0hJeQ\ne6xLl8b1q+kknQklcvRo43RIJT94/gPYvxCOrqBDpQ40KNWAr/Z/RVR8lNn6YmNlwbQuHnzYsgor\nD0TQ44e9xCbocr6aphmZM5CEA2XSvXcDLqRvICJXROT2OsL/AXVNP78K7BaROBGJA/4GGt7eTynl\nCViJyH5zdT4vKNiwISWHDuXG+vXE/PCDcWPz0eBWH1YPRl0NZVyjcSSnJjNpzySzzr0rpRjkW5np\nXT3Zdy6GTnN3Ehaj61NommbeQBIIVFZKlVdK2WAcQaxO30Ap5ZLu7cvA7emr80AzpZSVUsoaaJbu\nMzBOaz1wNJKfFO/di8Jt2nDpy+nE79oFltbQ+XuwsIRlvShjX5L3vd5nc9hmNp7baPb+dKzjxo+9\nG3DpeiKvztnJ4fBrZj+npml5m9kCiYikAAOA9RiDwG8ickwpNVEp9bKp2SDTEt9DwCCgp2n7cuA0\ncAQ4BBwSkTXpDt+VZySQKKUoPemzO2V6IyKgqDu88g1EBsGm8XSv3p1qxavx3z3/JfZW7KMPmkWN\nKpZgZX8f7KwteO3b3Ww49mSFtzRNyx/0A4lPiVuhoZzt0hUbd3fK/vwTFnZ28NdHsPdb6LaUEMdy\ndFvbjVcqvcIEnwk50qfoG7fosyiQwxGxjG1fnV6N9Y1kTctP8sQDiVr2sS1fntJTppAYHHynTG+r\nT8HFE1a9RzWrwvSo0YOVJ1eyJ3JPjvTJqbAtS99pRMtqzkxYE8yENccw6LrwmvbM0YHkKVK4RXMc\n+/cn9vffufbrr2BlC50XgCEZlr/NezX7UqZwGSbsmkBCSkKO9KmAjSVz36xL78blWbDjLP10wkdN\ne+boQPKUcRzwPgWbNeXipP9y8+BBKFER2s+AsN3Ybf+K8Y3GE3YjjLmH5uZYnywtFGNfqs64l6qz\nKSSKbvN3E33j1qN31DQtX9CB5CmjLCxwnTIF61KliBg8hJToaPDoArW7w7YvqX8znk6VO/HjsR8J\nvhKco33r1bg887vX40RUHK/O2cGpS7pQlqY9C3QgeQpZOjjg9vVsDDduEP6BqUxv2yngVBVWvsMH\nz3WnmF0xxu00PmOSk24nfExMTqXjHJ3wUdOeBTqQPKXsqlbF5dNPSdi3n6gpU8HG3ni/5NZ1HP4c\nysfeozgec5zFwYtzvG8ebkX5vb8PJXXCR017JuhA8hRzaN+O4j16cHXxYmJXrwbn6saRyZnNtAw/\ngq+7L3OC5nD++vkc71uZ4vaseM8H73LFGbrsEDM3ndRZbzUtn9KB5ClXcthQ7L29iRw7jsSQEKjz\nFtTsBAGT+NitDdYW1kzYNSFXfonfTvjYqY4bX206wbBlh0lKSc3xfmiaZl46kDzllLU1rl9Nx9LB\ngfCBgzDExhpXcRV1p+SaoXzo8S57L+7l91O5Uyr0dsLHD/yqsOJAuE74qGn5kA4k+YCVoyNus2aS\nEhVFxLDhiHVB6LIA4qLodHgddZ3rMm3fNKJvRudK/5RSDParzJddjAkfO+uEj5qWrzwykCilKiql\nbE0/v6CUGqSUKmr+rmmPo4CnJ86ffEL89u1Ez54NpWtDq0+xOPE34+2rcivlFp/v/fzRBzKjTnXd\nWNS7Phd1wkdNy1cyMyJZARhMNUK+B8oDP5u1V9oTKfZaV4p26cyVed9yY9MmaNAPqr5IuX++5L3y\nL7Px3Eb8z/nnah99Kjqy8j0fbK2MCR83BpuvjoqmaTkjM4Ek1ZTJ91Vghoh8ALg8Yh8tlzh/8gl2\ntWpxYcRIboWeNWYJLuRMj/0rqVq0EpP2TOJ60vVc7WNl58L8/r4PVZwL8c7ifSzcYZ6685qm5YzM\nBJJkpVQ3oAfwp2mb9UPaa7nodpleZWtL+MCBGFJtofP3WF8LY0KiDVcSrzBj/4zc7iYlC9ux9J1G\n+FVzZvyaYCauCdYJHzXtKZWZQNILaARMEpFQpVR5YIl5u6VlhbWLC67Tp5MUGkrkxx8jZRpAi9HU\nCFlH9+J1WHZiGYEXA3O7mxSwsWTem3Xp1bgcP+wI5b0l+0lIMuR2tzRNe0yPDCQiEiwig0TkF6VU\nMaCwiEzOgb5pWVCwYQNKDhvGjQ0biPn+e2j8AVRoTv9Df+FaoCQTdk3gliH3EytaWijGvVSDcS9V\nZ2NIFK/P36UTPmraUyYzq7b+UUoVUUoVx1itcIFSarr5u6ZlVfFePSnyYlsuTf+KuF27oON87G2K\nMC4mlnPXz/HtoW9zu4tpejUuz7dv1uXfqBs64aOmPWUyM7XlICLXgY7AAhGpC/iZt1tadlBK4fLZ\nZ9hWrMiFocNIupYMHefT6OIpXrFxZsHRBfwb829udzNNqxql+PWdRjrho6Y9ZTITSKyUUi4Y66T/\n+ajGWt5iYW+P2+xZiMFAxKBBpLo2giZDGXZyP0UsbRm3cxwpqXmnEJVnmbsTPq48oBM+alpel5lA\nMhFYD5wWkUClVAXgpHm7pWUnm3LlKD3lC2OZ3vETkGYjKerWgFGXojh25Rg/hfyU2128y+2Ej/XK\nFufD33TCR03L6zJzs32ZiHiIyHum92dEpJP5u6Zlp8LNm+P4/vvErlrFtWXLodN3tE4Smhms+Prg\nbMJuhOV2F+/iUMCaRb3r07GOq074qGl5XGZutrsppX5XSl1SSkUppVYopdxyonNa9nJ8vz+FmjXj\n4n8/5+bpaFSHuXwScQ7L1BQm7pqY5/7qt7Gy4Msungzxq8yKA+H0XKATPmpaXpSZqa0FwGqgNOAK\nrDFt054yysKC0lOnYO3iQsTgwSQXq0sp73cZEh3N7sjdrD69Ore7eB+lFEP8qvBlF08CzxoTPoZf\n1QkfNS0vyUwgcRKRBSKSYnotBJzM3C/NTCyLFMFt9mwMcXFEDPkAafYJXQtXonZSClP2TuZyQt5c\nKdWprhuLehkTPnb4Rid81LS8JDOB5LJS6k2llKXp9SZwJTMHV0q1UUr9q5Q6pZQamcHnPZVS0Uqp\nINOrT7rPpiiljimlQpRSs5RSyrTdRik1Xyl1Qil1XCml79c8JruqVXD57FMSDhwg6ssZWHRewPir\n8SQkxTFlT9591tSn0t0JHzfphI+alidkJpD0xrj09yIQCXTGmDbloZRSlsA3QFugOtBNKVU9g6a/\nioiX6fWdaV8foDHgAdQEvIFmpvajgUsiUsV03C2ZuAbtHg7t2lG8Z0+uLllC7LYjVHjxK965do2/\nz63n64Nf50p53sy4nfCxsk74qGl5RmZWbZ0XkZdFxElESopIB4wPJz5KfeCUaZVXErAUeCWT/RLA\nDrABbDEmibz952dv4HNT31JFJG/OxTwFSg4bin39+sYyvRbVeLtCBxrfTODbw9/S7vd2dFzdkTlB\nc/g35t88dSPemPCxIb464aOm5QlPWiHxw0y0cQXSrykNN227Vyel1GGl1HKlVBkAEdkFbMY4AooE\n1otISLqCWp8qpQ4opZYppZwzOrlS6h2l1D6l1L7o6NypDJjXKSsrY5neYsUIHzgIC5/RzCvwHOvD\nIvjoahxFrkcx79A8Oq/pTLvf2zF933QORR8iVXJ/Ga69jZVO+KhpecSTBhL1hG3u/bNxDVBORDyA\nTcAiAFMRrWqAG8bg00Ip1RSwMm3bISJ1gF3AtIxOLiLzRaSeiNRzctJrAx7EqkQJY5neS5eIGPEJ\n0v0PSvf2p3vFV1h4/hwB58IYl2iFuwEWBy/mzb/epOWylkzaPYk9kXty9an42wkfx7Y3JXz8326d\n8FHTcoF6kikLpdR5EXF/RJtGwHgRaW16PwpARDKs92q6pxIjIg5KqeGAnYh8avpsLJAITAXiMGYg\nTjWNYNaJSI2H9aVevXqyb9++x7vIZ8zVZcu4OGYsJd59l5IfDDFuTIqHY7/D/kUQvpfrljZsrdgQ\n/yJF2H7tBImGRBxsHWhepjl+7n40LN0QW0vbXOn/hmMXGbT0II6FbFnYy5tKJQvnSj80LT9RSu0X\nkXqPbPegQKKUusH9IwgwjjQKiIjVIzpgBZwAfIEIIBB4Q0SOpWvjIiKRpp9fBUaISEOl1GtAX6CN\n6XzrMFZnXKOUWgrMF5EApVRPoJ2IdHlYX3QgyZzIMWO5tmwZpb+YjMMr99zOigqGg4vh0C+QcJWE\nou7srPICm+ws2XJxLzeSb2BvZU9Tt6b4lvWliWsTCloXzNH+Hwq7xtuLAklKSeXb7vVoVLFEjp5f\n0/KbLAeSbOrEi8AMwBL4QUQmKaUmAvtEZLVS6nPgZSAFiAHeE5HjptHJHKApxmC2TkQ+NB2zLLAY\nKApEA71E5KFLjHQgyZzUpCTC+vTl5t69lOj3Lk6DBqEs7pn9TE6E43/CgUUQuhWUBcmVWrK3YiM2\nGWIJCN9MTGIMNhY2+JT2oYV7C5qXaU5Ru6IZnzSbhcXcpNfCQM5diWdKZw9era2TMGjak8oTgSSv\n0IEk8yQpiciJE4ldvoJCfr64fvEFFgUfMLK4cto4Sjn4E8RfgsIuGDy7EVSuHptijuJ/3p/I+Egs\nlSX1nOvhW9aXFmVa4Fwww/UR2Sb2ZjLvLtnH7jMxfOBXhUG+lTA9hqRp2mPQgSQdHUgej4hwdfES\noiZPxrZSJdzmzMHGLaMFdyaGZDix3jhKObUJJBUqvIDUfovgkhXxj9jKxnMbOXv9LAAeTh74ufvh\n5+5HmSJlzHINSSmpjFxxmJUHI+hc143/vloLG6snXVuiac8mHUjS0YHkycTt2EHEBx+iLC1xmzUT\ne2/vR+8UG24coRxcDLFhYF8CPLtBnbc4Y23NpvOb2HRuEyExIQBUKVYFP3c/fMv6Urlo5WwdOYgI\nM/1PMmPTSXwqlmDum3VxKGCdbcfXtPxOB5J0dCB5crdCQwnv/z5JYWGUGjuGYl27Zm7HVAOc2Wxc\n8fXvX5CaAu6NoM5bUL0DEUlX8T/nj/95fw5eOogguBd2x7esL37uftR0rImFyp4RxPL94YxccZjy\njgVZ0Msbt2L22XJcTcvvsi2QPGD1ViywDxgqImeeuJc5RAeSrDFcv07E0GHEb9tGsf/8B+dRI1FW\nD120d7e4S8bVXvsXQcxpsHUAjy7GoOLiyeWEywScD8D/vD97I/eSIimUtC+Jr7sxqNRxroOVxWOc\nLwM7T13m3SX7sbO25Pse9fBwy5mb/5r2NMvOQDIBuAD8jHEp7utAKeBfjKusXshyb81MB5KsE4OB\nS1OnEbNwIfaNGuL21VdYFn3MX8YicG4HHPgRjq0Cwy1w8YK6PaBmZ7ArQuytWLaGb2XTuU3suLCD\nW4ZbFLUtanxWpawfDV0aYmNp80TXcDLqBj0XBBITn8TsbrXxq27em/6a9rTLzkCyR0Qa3LNtt+l5\nj0Mi4pnFvpqdDiTZ59rK37k4bhxWpV0oM2cOthUrPtmBEq7C4d+Mo5RLx8DaHmp0NAYVN29QipvJ\nN9l5YScbz21ka/hW4pLjKGhdkKaud55Vsbd+vGmqSzcS6bNoH0cjYhn3Ug16+JR7sv5r2jMgOwPJ\nLuArYLlpU2fgQ1MgCRIRryz31sx0IMleNw8cJHzQICQhAdfpX1KoWbNH7/QgIhBxAA4shCMrIDke\nnKoZp708Xwf74gAkGZLYE7kH//P+bA5L96yKqw9+7n68UOYFHGwdMtf/pBQG/RLEppAo3n6+PB+/\nWA1LC708WNPulZ2BpAIwE2hk2rQL+ADj0+p1RWR7FvtqdjqQZL/kyEjC3n+fWyHHKTlsKMV79876\niqtbN+DoSuMy4oj9YGkD1V6COj2gXBMwPRxpSDVw8NJB/M/7s+n8Ji7GX8RSWeJdyhs/dz9auLfA\nyf7h+dUMqcKnfwazcOdZWtdwZsZrtSlgY5m1/mtaPqNXbaWjA4l5pN68yYWPR3Nj3TocXnmZUhMn\nYmGbTbm2Lh413ks5vBQSY6FYeajTHbz+A4VLpTUTEYKvBKctKz57/SwKhaeTJ35ljUGlTOEHP6vy\nw/ZQPl0bTFXnwnzbvS5lS+RsWhdNy8uyc0TiBszGWGhKgO3AYBEJz46O5gQdSMxHRLg8dy6XZ83G\nztMDt9mzsS5ZMvtOkJwAIWuM91LObQdlCVXbGqe+KvmBxZ1RhIhwJvYMm85twv+8f9qzKs8Vfy5t\nBVjFohXvGzn98+8lBi8NIlWEGa954VtN34TXNMjeQLIR44qtxaZNbwL/EZGWWe5lDtGBxPyub9jA\nhREjjTXhv/6aArVqZv9JLp+Cgz9C0M8QHw1FXKH2m8ZX0fuTUYffCMf/vPFZlaBLQQhCuSLljEGl\nrB81StRICyphMTfpt2Q/xy5cZ1CLSgz2q6Lvm2jPvOwMJPfdUH9abrLfpgNJzkg8fpzw/u+TcuUK\nLv+dhEO7duY5UUoSnPjbOPV1yt+4rWIL44qvKm3B6v7lwdE3o9kctplN5zYReDGQFEnB2d4Zv7J+\n+Lr7UrtkbVIMijGrjrJsfzhNqzgx8zUvihV8sqXGmpYfZGcg2QQsBH4xbeqGMeOub1Y7mVN0IMk5\nKVeuED54MAn79j84g3B2unYeDi4xvq5HQEEnU0qWHuBYKcNdYm/FsiV8C5vObWLnhZ3cMtyimG0x\nmpVpRvMyzQmLKMOkP0/hVNiWeW/WpZZb5laDaVp+k52BxB34GuOqLQF2AoMelbo9L9GBJGdJUhIX\nP/2Ua8uWPzqDcHZJNRhHJwcWwb9/gxigbGNjQKn+MlgXyHC3m8k32R6xnYCwALaGbeVG8g0KWBWg\nZrH6HDnhRmxMZT57qT5dvc2TXFLT8jKzrtpSSg0RkRlP1LNcoANJzrs/g/A32LjlUG2QG1EQ9JNx\n6utqKNg5gMdrxqBS6sH3bpINyQRGBRJwPoDN5zdzKeESiAUpNyvgWbwxX7R9g7IOpXPmGjQtDzB3\nIHlkqd28RAeS3JM+g7DrzBkUrF8/506emgpntxkDSshqMCSBa13jiq+ancD2weV4UyWVY5ePsem8\nP8tD1nHdEAFA5aLVaFPeeF+lgkMFXedEy9fMHUjCROSpGevrQJK77sogPGYMxV7LZAbh7HQzBg4t\nNU59RR8H64JQq5NxlOJaFx4REBbv28uUrSug0FGwNc7qli1SlhZlWtDCvQUeTh7Zlq1Y0/IKPSJJ\nRweS3Ge4cYOID4feySA8cgTKOhdqg4hAeKDxuZRjKyH5JpSsAfX7gMfrYPPg3F1nouPot2Q/p2Mu\n0Nr7Cil2Rwi8aMxWXMKuBM3dm+Pr7kv9UvWfOLGkpuUlWQ4kD0gfD8YMwAVEJGt5vXOQDiR5gxgM\nXJr2JTELFmDfsCFuM54gg3B2SrwOR5fDvgVw8TDYFYW6PaF+X3DI+H7OzaQURq44wupDF/Cr5sz4\nDhU4fGU3/uf92R6xnZspNyloXZAmrk3wdffledfnKWRTKGevS9OyiU6Rko4OJHnLtd9XcXHsWKxc\nXCgz5xtsK2W8TDfHiMD53bB7Dhz/E1DGHF8N+0OZ+vdNe4kIC3eeZdLaENyKFWBe97o8V6oItwy3\n2BO5x3iz3pRY0srCigYuDfB196V5meY4FnDMnWvUtCegA0k6OpDkPTcPHiR8oDGDcOkvp1H4hRdy\nu0tG187D3vnGG/SJsVC6tjGgVO9w34OOgWdj6P/TAeISU5jcqRaveN2pa29INXAo+lBawa7wuHAU\nCg8nD3zdfWnh3oKyRcrm9NVp2mPRgSQdHUjypuTISMLfH0BiSAglh35I8bffzjuroG7FGas67vkW\nrpyEQqXAuw/U6wUF74wqLl1PZMDPB9l7NoaePuX4+MVq2FjdfdNdRDh57SQB5wMIOB+QlgOsUtFK\nNC9jvK9SvUT1vHPtmmaiA0k6OpDkXakJCVz4+GNu/G2GDMLZITUVTgcYp71O+4OlLdTqAg37Qala\nACQbUpn893G+3x5K3bLFmPOfOjgXsXvgIS/EXWBz2GYCzgewP2o/BjHgbO9MC3fjCrC6znWxtsiF\nhQiado88EUiUUm0w1jKxBL4Tkcn3fN4TmIqxtgnA1yLynemzKUA7wALYiDHjsCil/gFcgATTPq1E\n5NLD+qEDSd4mIlyZN4/ombOw8/DA7etsziCcXaL/hT3zjMuIk28aa6Q0fA+qtAELS9YcusCIFYex\nt7Himzdq06BCiUce8lriNbaEbyHgfAA7L+wk0ZBIEZsiNHNrRgv3FviU9nnsKpCall1yPZAopSyB\nE0BLIBwIBLqJSHC6Nj2BeiIy4J59fTAGmKamTduBUSLyjymQDBORTEcGHUieDmkZhAsXxu2bb8yT\nQTg7JFw13kPZMx+uh0OxclD/Xaj9JidiFf0W7+dczE1GtX2Ot58vn+kpq4SUBHZe2EnA+QD+CfuH\n60nXsbW0pVHpRrQo04IXyrxAMbti5r02TUsnLwSSRsB4EWltej8KQEQ+T9emJxkHkkYY83s9j3G5\n8Vagu4iE6ECSv+VYBuHsYEgxrvLaPRfCdoNNIfD6D3FevRnqH8f6Y1G083BhSicPCto+3mr5lNQU\nDkQdwP+8PwFhAVyMv4iFsqBOyTppU2CuhVwffSBNy4K8EEg6A21EpI/pfXegQfqgYQoknwPRGEcv\nH4hImOmzaUAfjIHkaxEZbdr+D1ACMAArgM/kERehA8nT5a4Mwu++i9NgM2cQzg4RB4zTXkdXQmoK\nUqUVawp0YMjeIlRwKsy8N+tSqeSTPU8iIoTEhKStADt17RRgLNh1+8n6KsWq6Jv1WrbLC4GkC9D6\nnkBSX0QGpmtTAogTkVtKqX5AVxFpoZSqhPHeymumphuBESKyVSnlKiIRSqnCGAPJEhH5MYPzvwO8\nA+Du7l733LlzZrlOzTzuyiDs60vpL77AstBTUAb3xkUI/B72/QA3LxNftApfxrbgj9TnmdSlHm1q\numT5FOevnzeuAAsLSCvY5VrIlRbuLfB198XLyQtLC11/Xsu6vBBIHjm1dU97SyBGRByUUsMBOxH5\n1PTZWCBRRKbcs09PMpgau5cekTydRISrS34yZhCuUAG3uXNyLoNwViUnwtEVxmmvqCNcV0VYnPwC\nqd59eK99E6wss2eEdTnhMv+E/UPA+QB2R+4mOTWZ4nbFaebWDF93XxqWboitZR5aBac9VfJCILHC\nOF3li3FVViDwhogcS9fGRUQiTT+/inHU0VAp9RrQF2iDcWprHTAD+BsoKiKXlVLWGIttbRKReQ/r\niw4kT7f4nTsJH/IByoEheWYAACAASURBVMLi/+3deXxU5d338c8vk50sZCNAyEIiIktYwiKIEqpW\nEAluoMimba3F1ts+tvZWa+/7afWudvGubZ+61aU1IOKubIJaRaiirIGEnayEJQvZ98zM9fxxBhgQ\nJTCZLOb3fr3yIpM5Z+a6mGS+c65znd9F3F//0rEVhD1lDBR+hmPj08i+1TiMD5uDr2DoTQ/Qe9Bl\n7fpU9a31bDi8gY+LPmZD8QbqWusI8g3i8rjLuTLhSiYPmEyYf1i7Pmd30dji4EBpLcP6h+sSyueh\n04PE1YjpWAFgA14yxvxWRB4BthhjlovI48BMwA5UAHcbY/a6jk6expq1ZYA1xpifiUgvrBPvfq7H\n/Aj4mTHG8U3t0CDp/loKCjj045/QUlRE31/9iog5t557p66mIp/9K/5Ev7w3CJVG6mNG0Wvyf8DQ\n68HWvteNtDpa2XRs08lyLWWNZfiKL+P6juPKhCv5Tvx3iO0V267P2RUVlNez5ItCXt9yiJomO8nR\nvbhrcjI3psUR4KvDf+fSJYKkq9Ag+XZw1NZy+Oc/p379BiLmziX2oQc7p4Kwh/YUHGbtK3/ihpaV\nJMkxTGh/ZNwPYMz3oNe5rz05X07jJLs8++SV9QU1BQCMiR1DRnIG30367rfqSMXhNHy6v5TMjYWs\n21eGr48wdXhfJiZHsWxzETmHa+gTGsAPLh/I3EsTCA3sfr9DHUWDxI0GybeHcTgo/d8/UfHSSwRP\nmEDck3/CN6L7XVtR3dDKfcu2Yg5+xIMRHzO4fiv4BsKIW+DSuyF2qNeeO68qjw8KP2BV3ioKagrw\n9/FnSvwUMlIymNR/En7tfHTUUaoaWnh9yyGWfFFEUUUDMaEBzB2fwNxLE05WGjDG8NnB4zzz6UE+\nO3ic0EBf5k9I5HuTkugT+vXVCHoqDRI3GiTfPl2ugvAFcDoNf/34AH/51wGuiankifiNhO57E+xN\nMDDdump+0FRop6nPxuGg+cABGrZtoyU3j15T0ikYHM7K/FW8n/8+lc2V9A7ozbSkaWSkZJAandot\nphTnHK4mc2MB72UdodnuZHxSJAsmJjJ1WN+v1D1zt7O4iuc+zWN1zlH8bD7MGjOAu65IJim6G8wO\n7CAaJG40SL6dTqsg/MQfCf3Odzq7SRfkk72l/HTZdgD+dmMik2tWw6bnofYIRCa7rpqf941LA5+N\ns6GBxp07adi2jcZt22nMysJZV2fd6ecHra34JycTuXABwTOu5YvKLFbkreCTok9ocbaQFJbEdcnX\ncV3ydcSHdq0FUZvtDt7PPkbmxgK2FVUR5GfjhtFxLJyYyJB+5zdMl19ez9/X5/HW1mLsTifXDu/H\novQUUgeEe6fx3YgGiRsNkm8v9wrCMT+7j6g77+wWn6LPVHS8gUVLtrL7aA33XjWIn05JwrZvhTV9\nuHgzBITB6Pkw/i6IHHjWx2gtKaFx2zYatm2ncds2mvbuBYcDRAi46CKC0tIIThtNUFoavrGx1L7/\nPhUvZ9K0ezc+4eFE3DKbiLlzaYoK4cPCD1mZt5LNxzYDMLrPaGYkz2Bq0lTCAzrvDfZIVSNLvyxi\n2eYiyutaGBjdi/kTEpk1ZgDhQZ4NyZXWNPHSZwW88kUhtc12Lr8omrunpHBZSlS3/J1qDxokbjRI\nvt3cKwiHzcyg36OPdq0Kwm3U1Org4XdyeGtbMekXx/CXOaPoHewPxVusQNn9LjgdMHg6ZtyPaLbH\nnjra2LaN1iNHAJDAQIJSUwkak0ZwWhpBo0ZhCzv7p3RjDI3btlHxcia1H30EIoRNvYbIhQsJGjWK\no3VHWZW/ihW5K8irzsPPx4/JAyaTkZzBFQOu6JAlhY0xbMw7TubnhXy4pwSnMVx1SR8WTEziioui\n8Wnn6bw1Ta0s/bKIF/+dT1ltM6lx4SxKT2Ha8L49buqwBokbDZJvv25TQfgcjDEs3VTEr5fvIjYs\nkGfnj2F4XLg1TLXxExref5nG7Vk0lgrOVmv83xYdbQVG2miC09IIHDLkgmaztRQfpnLpUqreeANn\nbS2BI0YQuXAhYVOvAV9f9lTsYUXuClbnr6aiqYIw/7CT51NGxoxs90/tdc123t5WzOKNhRworaN3\nsB+3jotn/qWJxEd6vyJyU6uDd7Yf5u/r88gvrycpKpi7JqdwU1ocgX49Y+qwBokbDZKeo+bDD60K\nwiEhXbuC8Dls37affzz3HgMOH+AqxzGCi/JODVOlpBCUGEqwzz6CAvLxi4lAxn0Pxv4AwjwvweKs\nr6fq3XepzFxMS2Ehvn36EDF3Lr1vvQXfiAjsTjsbj2w8eT6lydHEgJABzEiZQUZyBglhCR49/8HS\nWjI3FvL2tsPUNdtJjQtn4cREMkb275Q3cIfT8MGuYzzzaS47i6uJDgng+5cnMX9CImHf8qnDGiRu\nNEh6lqZ9+yi++8dWBeHf/pbwGV24gjCu2VQHD552fqP1sLVET6uvP3vCB0DqSK655buEjUnDFu46\nR2EM5K+3ikXuex98fGHYjdaiW3FjPG+X00n9hg1UvJxJ/eefIwEBhM+cSeTCBQQMGgRYV9N/VPgR\nK/JWsOnoJgyGETEjyEjOYGrS1DaXvbc7nHy0p5TMjQV8nnscf5sPM0b0Y8HEREbF9+4S5yiMMWzM\nPc4zn+ay4UA5oQG+zJ2QwA8mDaTPNyxk1p1pkLjRIOl57BUVFN97b5esIPxNs6lsMdEEjz41TOV7\n8WD+tC6fZ9blMnJAOE/PH0Nc76CvPujxXGum1/Yl0FIL8ZfCpYtgyEywnV8J+7Np2r+fysVLqF6+\nHNPcTK/LLiPy9oX0uuKKk/+vx+qPsTp/NStyV3Cw6iC+4svlAy4nIzmD9Pj0s9b8Kq9r5rXNh3jl\ni0KOVDfRPzyQeRMSuXVcPNEhXfc8V87hap79NJfV2Ufx9fHh5jFx3DU5hYHfsqnDGiRuNEh6JquC\n8P9Q9cYbhFx5Jf3/8IdOqSDcWlJK4/ZtNGzddvpsKiBg0EUEjU4jeEwaQWlp+A0YcNZP32tyjnH/\nGzvw9/Xhr3NGc/mg6K9sA0BTDWS9Yh2lVBZA2AAYfyek3Q7BkR73xV5ZSdXrb1D5yivYS0vxT0oi\nYsF8et9wAz69rP9bYwz7K/efPJ9S1lhGqF8o1yRdQ0ZKBqNiRrGjuIbMzwtYnX2MFoeTyy+KZsHE\nRK66pE+7FbTsCIXH63l+Qx6vbymm1eFk2rC+LEpPYWR8785uWrvQIHGjQdJzdXQF4W8apjo5m+rE\nNNxRo04NU7VBXlkdP1q8ldyyOu6fOpi701O+fsjH6YD9a6215gs2gG8QjJxjHaX0ucTzfra2UrP2\nAyoyM2nauROf0FB6z5pFxLx5+A84teCWw+ngy6NfsiJvBR8VfkSTowmbM4qGihH4N45j1og05k9I\nvOC1WrqKstpm/vl5PpkbC6ltsnNZShSL0lO4YlB0lxiWu1AaJG40SNRpFYT/8hd6Xdo+FYTPZ5gq\n8JJLEH/PpsvWN9t54K2drNx5lGuGxvLELSPPfcL3WA58+QzsfAMczZBypXU9Sv80COkDHr7RNWZl\nUZGZSc3aD8AYQq++msjbFxKUloaIcKiigSVfFLJs60HqbTsIj9lBq/9+DE6GRw1nRsoMpiVNIyqo\n/euMdbTaplZe3WRNHS6paWZY/zAWpadw7fC+3epI6wQNEjcaJArOrCD8MBFz5pz3Y5wcpnIFR9Oe\nPV8ZpjoRHH7x8V75NGqM4aXPCnhs9R4SIoN5dv4YBvdtw1Xv9eWw9R+w6QWoO2b9zDcQwgdA7wQI\nj7f+PfEVHg+hfaGNi2S1Hj1K5dKlVL7+Bs7qalpTBvPBkCk8ZxuI0+bHNUNjWTgxiQnJkZQ1lvF+\n/vuszFvJ3oq92MTGpLhJZCRnMCV+CoG+3fvkdbPdwbvbD/Pc+jzyyupJiAzmrsnJzBozoFtNHdYg\ncaNBok44vYLwbcQ+9NDXXnPhzWGq9rApv4KfLN1GXZOd392cyvWj2riGu73Fmu1VkQfVRVBVBFWH\nrH8byk/f1scPwuNcwXIiZOJPBU1Y3Gkn86sbWnnr8wPkvfomV+z4Fwl1pTSFRdB7zhzib5+Hb9RX\njzoOVB5gRd4KVuWtorShlF5+vfhu4nfJSM5gbN+x+Ej3+yR/gtNp+GB3Cc98msuOQ1VEh/jzvUkD\nmT8h0eMr8TuCBokbDRLl7rQKwpdeStyfn8Q3IsI1TJV96sS4+zBVdDTBo63yIsFj2meYqj2U1jTx\nk6Xb2FxQyfcmJfHL6UPw82QIpaUeqotdwVII1YdOD5oTRzIniA3C+lMf3J99jb35oiKEAkckvWIG\nMiltJGPsTdS++jr1GzYg/v6EZcwgcuFCAgcP/spTO5wONpdsZmXuSj4s/JAGewN9e/XluoHXkZGS\nQUrvlAvvVyczxvBFXgXPfprLp/vL6OVvY96ERL4/aSB9w7vu0ZcGiRsNEnU2JysIx8Rgi4zslGGq\n9tDqcPLY6j3847MCxiZG8PS8NO9d19DaBDWHoaoIe0UhuQf3UFK4n4CGI8RLGX2lEh+cbjsIhPal\n2d6Xil1QnVWOaXEQPGIQkbfNJmTaDUjQV4flGu2NfFL0CSvyVrDxyEYcxsGQyCHMSJ7B9OTpRAd9\nzay1bmDXkWqe+zSPlTuPYPMRbhxtTR3uihMONEjcaJCor9OYlcXRX/8GW2hopw5TtYf3sg7z4FvZ\nhAT68tTcNMYP9Hy679mU1DTxypdFvLqpiLLaZhIig1kwIZHZYwfQO0BcQeM6gjl5RGN9OcqOUHUw\ngIoDwdgbfPELsRM5XAgf2w9bnyTXsFmi2/maeMqdzSfPp+w+vhub2JjQfwIZyRlcmXAlQb5nua6m\nGzhU0cDzG/J4bfMhWhxOrhkay6L0FEYndJ31dTRI3GiQqJ5i37FaFi3ZSlFFA7+cPoTvT0pqlyMp\nYwyb8ivI3FjI2l3HcBjDlItjWDgxifSLY9peONHpgNqjmOMF1H7wARUrPqUxtxSfAB96DxYiksrx\nD248fZ+giJPBkhsSwUpTy8q6XI61VBPsG8TVCVczIyWD8X3HY2vjxICupLyumZc/LyBzYyHVja1c\nOjCSu6ekkH5xTKcfBWuQuNEgUT1JTVMrP399Bx/uLmHGiH78/uYR9Aq4sKvb65vtvLP9MIs3FrKv\npJbwID9uGTuA+RMSSYxqn4s7G3fupCJzMTVr1oDDQUj6JCJnTCY4IRCpKT79HE31IWhtwAlsDQxg\nRUgvPuzVizofoQ++XBcUz3XRoxkcO/rU0U1wlMdTnDtCXbOdZZuKeGFDPsdqmhjSL4xF6clcl9qv\n06YOa5C40SBRPY3TaXh2fS5PrN1HSkwIzy4YQ0pM28fgc8vqWLyxkLe2FlPbbGdovzBuvyyRmSPj\nCPL3zqf+1pJSKl9dStWy13BUVRFwySVW9eHrpp9aFsAYaDh+aris+hBNlfmsq9zDypZjfObTil2E\ni5tbyKirZ3p9A318AqwjmqgUGHoDDMkAf+9XD75QLXYn72VZU4cPltYRHxnED69IZvaYeK/9338d\nDRI3GiSqp/r3gXLuXbadFruTJ2aPZNrwvl+7rcNp+HivVThxw4Fy/GzC9NR+LJyYSFpCRIcNszib\nmqhesYLKzMU0HziALSqKiDlziJhzK74xMd+4b0VTBWv2v8PK3BVk1+Tig3CpfxQZziCuKj1EcHWR\ntUjYsButhcIGjOuyRytOp+GjPSU8+2ku24qqiOrlzx2XJbFgYqK1Tk0H0CBxo0GierLDVY38eMlW\ndhRXsyg9hfuvufi0oZKK+haWbS7ilS+KOFzVSN+wQOZdmsCc8QnEhHZe4URjDA1ffEHFy5nUrVsH\nfn6ET59OxMIFBA0bds7986vzWZm3klV5qzhcdxh/H3+GhMSR2tTE8KP7SW2oJT48CRk1D0be1i4l\n+L3BGMPmgkqeWXeQT/aVEexv47bxCdx5xUD6hXt3ooEGiRsNEtXTNdsd/GbFbpZ+WcRlKVH89bbR\nHK5s5OWNBazceZQWu5MJyZHcPjGJq4fGenYtihe0FBRQseQVqt5+G9PQQPDYsUTcvpDQK69EbN88\n3OM0TrJKs/hX0b/IKc9h9/HdNDmaAAg3wvDGBlJbWkmNHMKw1HlEDb8FfLtm5eG9x2p47tM8lu84\ngo/A9aPiWJSezEV92lDZ4AJokLjRIFHK8vqWQ/zq3Rx8BJpanQT727gpLY6FE5O4ONY7b0btyVFT\nQ9Vbb1O5eDGtR47gFxdHxPz59J51M7bQtrXf7rSTW5VLdnk2OeU5ZB/bysHagpNXv8TZnQzv1Z/U\nhO8wPPkahkQNJdiva51TOVTRwIv/zmfZ5iKaWp181zV1eExi+04d7hJBIiLTgL8ANuAFY8zvzrj/\nDuCPwGHXj/5mjHnBdd8fgOsAH+BD4KfGrbEishxINsaccwk8DRKlTjmxlsbYxAhuGjOgW67yZ+x2\naj/+mMrMxTRs2YJPcDDhN91E5Px5+CclnffjNbQ2sKd8Fzn73iW7eAM5zeUc9rWOdHwQLgofSGqf\n0QyPHk5qdCopvVPw9fF8nRdPVdS38M/PC8jcWEBVQyvjk6ypw1MGt8/U4U4PEhGxAfuB7wLFwGbg\nNmPMbrdt7gDGGmPuOWPfy7ACZrLrR/8GHjLGrHPdfxMwCxihQaJUz9a4axeVmYupXr0a7HZC0tOJ\nvH0hwRMmXPibaUMF5VmL2bX7NbLriskJDCA7qBc1WJUPAm2BDI0aejJYhkcPJy4krtOu+6hvtvPa\n5kO8sCGPI9VNXNI3lB+lJzNjRH+Phim7QpBMBH5tjJnquv0QgDHmcbdt7uDsQTIR+BtwOSDAemCB\nMWaPiIQAa4C7gNc1SJRSAPayMiqXvUblq6/iqKggYNAgIm9fSNiMGfgEelAypmQXZC3F7HyNQ82V\nZIdFk9PvErL9fdlTU0CLswWAiICIk8GSGpPK8Kjh9A7s2AWuWh1Olmcd4bn1uewvqSOudxBv//gy\nYi+wZE5XCJJZwDRjzJ2u2wuAS91DwxUkjwNlWEcv9xljDrnuewK4EytI/maMedj18yexgmU7sFKD\nRCnlztncTM2q1VRkZtK8dy+2iAh633oLEbfNxS+2z4U/sKMVDnwA21+BA2vBaae1/2gOXHINORH9\nya7JJac8h9yqXAzW+2p8aPypcIlO5ZLISzqkRL7TNZX7X3tLeOzG1As+UuoKQTIbmHpGkIw3xvyH\n2zZRQJ0xpllEFgG3GGOuFJGLsM6t3Ora9EPgAaAGeNQYkyEiSXxDkIjIXVhHLSQkJIwpLCz0RjeV\nUl2UMYaGzZupyMyk7l8fg68vkQsXEH333dhCPCyQWFcG2a9boVK6C2wBMGQGjJpHffw4dlfuI7s8\nm+yybLLLsylpKAHAJjYujrj4tCGx5PDkLlvapSsEyTmHts7Y3gZUGGPCReQXQKAx5lHXff8NNAG1\nwH8BLYAv0Af43Bgz5ZvaokckSvVsLYcOUf70M1S/8w626Gj63Hcf4TfegPh4OM3ZGDiaZQVK9hvQ\nVGWt0TLyNhg117qaHihtKCWnPMeaJVaeza7yXdS21gIQ5BvEsKhhJ4MlNTqVvr36dnqdLegaQeKL\nNVx1FdasrM3AXGPMLrdt+hljjrq+vxF4wBgzQURuBX4ITMMa2loD/NkYs8Jt3yR0aEspdR4as7Mp\n+Z/f0rhjB4HDhhH78MMEp41unwdvbYJ9qyHrFcj9GIwTEi6zAmXYDRBwanqy0zgprCk8GSw55Tns\nrdhLq7MVgKjAqNOCZVj0MMIDOr4idacHiasR04E/Y03/fckY81sReQTYYoxZLiKPAzMBO1AB3G2M\n2es6Onkaa9aWAdYYY352xmMnoUGilDpPxhhqVq6k9I9PYC8tJWzGDPrc/3P8+n59+ZjzVnMEdrwK\nWUvh+EHw6wVDr4fR8yBx0lnLsrQ4Wthfuf/U9S3l2eRX55+8PyksieHRw0+Gy+DIwQTYvHvhZJcI\nkq5Cg0QpdSZnfT3lL7xAxYsvgc1G1A/vJOr73/dshteZjIFDmyBrCeS8Ay21EJEEJ8qy9I7/xt1r\nWmrYfXy3FSyu8y1ljWUA+Pr4Mjhi8Gkn85PCk9p1aWINEjcaJEqpr9NSXEzpH5+gdu1a/Pr3p89/\n/iehU69p/3MULfWwZwVsXwIFGwCB5HQYNd86Ue937rpZxhhKGkpOGxLLKc+hwd4AQIhfCMOihp12\nMj+2V+wFN1mDxI0GiVLqXOq/3ETJY4/RvG8fwePGEfvwLwm85BLvPFllAWS5hr6qiyAgHIbfZB2p\nDBh7XhWJHU4HBTUFpw2J7a/Yj93YAdhw64YLvp5Fg8SNBolSqi2M3U7Vm29S9ue/4Kipoffs2cT8\n9F58I72zbDFOJxT+25r1tfs9sDdC9GDrBP3IORB6Yedtmh3N7K3YS25VLjcNuumCm6dB4kaDRCl1\nPhzV1ZQ99RSVryzFJziYmHt+QsTcuYifF+uSNdXArnesWV+HvgSxwUVXWyfoL74WfDtmDRJ3GiRu\nNEiUUhei+eBBSh7/HfWffYZ/cjKxDz1IyBVXeP+Jyw9YgbJjGdQehaBIGHGLNfTVb4T3n99Fg8SN\nBolS6kIZY6hbt46S3/2O1sIiQtLT6fPgAwQMHOj9J3c6rGtSti+xrlFxtEDfVOsEfeps6BXl1afX\nIHGjQaKU8pSzpYXKxUsof/ppnC0tRC5YQPTdi9q8DorHGiog5y0rVI5mgY8fDJ5mhcpFV4Ot/cva\na5C40SBRSrUXe3k5pU8+SfXb72CLjKTPff+H8BtvPOdKje2qZJd1gn7na9BQDiGxMOJWax36mMHt\n9jQaJG40SJRS7a0xO4eSxx6jcft2AocOJfbhXxI8ZkzHNsLeYlUkzlp6siIxcWOtE/TDb4ZAz8qq\naJC40SBRSnmDMYaaVaspfeIJ7MeOETZ9On1+cT9+/fp1fGPqSmHn69ZJ+tLd4BsIQzJg2u8v+FyK\nBokbDRKllDc5Gxo4/sKLHH/xRRA5VW4l6NxXq7c7Y+DIditQ8tfD3Z+D7cKmLWuQuNEgUUp1hNbD\nhyl54glq31+Db/9+xP7iF4ROm9Z5JeGNOa+r5M/U1iBpv+peSinVw/nFxTHgySdJXJyJLbw3h+/7\nGYULFtC0e3fnNKiDAkyDRCml2lnwuHEMfPMN+v7mN7Tk5pF/8yyO/td/Yz9+vLOb5hUaJEop5QVi\nsxFx6y2krF1D5MKFVL3zDrlTp3H8H//EtLR0dvPalQaJUkp5kS0sjNiHHiR5+XsEjR5N6e9/T971\nN1C3fn1nN63daJAopVQHCEhOJuH5vxP/3LNgDIfu+hFFP/oRzXn55965i9MgUUqpDhSSnk7y8vfo\n88ADNG7dRt7MmZT87vc4amo6u2kXTINEKaU6mPj7E/W9O0hZu4beN95IxcsvkzvtWipffx3jcHR2\n886bBolSSnUS36go+j36CElvvoH/wIEc++//S/6s2TRs3tzZTTsvGiRKKdXJgoYNI3HJYuL+9L84\nqqooXLCQ4vvuo/XIkc5uWptokCilVBcgIoRNn07K6lVE33MPdZ+sI/fa6ZT9v7/hbGzs7OZ9Iw0S\npZTqQnyCgoi55yekrF5F6FVXUf7UU+ReO53qVavoqiWtNEiUUqoL8uvfn7g//S+JSxZji4zgyM/v\np3DefBpzdnV2077Cq0EiItNEZJ+IHBSRB89y/x0iUiYiWa6vO93u+4OI7BKRPSLyV3FVPRORNSKy\nw3XfsyLSgavJKKVUxwoeO5aBb7xB30cfoaWwkILZsznyq19hLy/v7Kad5LUgcb3BPwVcCwwFbhOR\noWfZ9DVjzCjX1wuufS8DJgEjgOHAOCDdtf0txpiRrp/HALO91QellOoKxGYjYvZsUta8T+Qdd1D9\n7nvkTruW4y/9o0uUW/HmEcl44KAxJs8Y0wIsA65v474GCAT8gQDADygBMMacuGrH13V/1xw0VEqp\ndmYLDSX2gf8keflygseMofQPfyAvYya169Z16vkTbwZJHHDI7Xax62dnullEdorImyISD2CM2Qh8\nAhx1fa01xuw5sYOIrAVKgVrgTS+1XymluqSA5IHEP/cs8X9/Dnx8KF50N4fu+hHNeXmd0h5vBsnZ\nCuGfGZkrgCRjzAjgI+BlABG5CBgCDMAKnytFZPLJBzFmKtAP62jlyrM+uchdIrJFRLaUlZV52hel\nlOpyQiZPJnn5e8Q+9CCNWVnkzbyekscf7/ByK94MkmIg3u32AOC0q2uMMceNMc2um88DY1zf3wh8\nYYypM8bUAe8DE87YtwlYztcMlxlj/m6MGWuMGRsTE+NxZ5RSqisSPz8ib7/dKrdy001UZC4md+o0\nKpe91mHlVrwZJJuBQSIyUET8gTlYb/wniUg/t5szgRPDV0VAuoj4iogf1on2PSIScmIfEfEFpgN7\nvdgHpZTqFnwjI+n3yG8Y+PZbBKSkcOzXvyb/5lm0lpR6/7m99cDGGLuI3AOsBWzAS8aYXSLyCLDF\nGLMcuFdEZgJ2oAK4w7X7m1hDVtlYw2FrjDErRCQWWC4iAa7H/Bh41lt9UEqp7iZwyBASFmdSu/YD\nalatxDc6yuvPKV31Ssn2NHbsWLNly5bOboZSSnUrIrLVGDP2XNvple1KKaU8okGilFLKIxokSiml\nPKJBopRSyiMaJEoppTyiQaKUUsojGiRKKaU8okGilFLKIz3igkQRKQMKL3D3aKDrrCDTMbTPPUNP\n63NP6y943udEY8w5ixX2iCDxhIhsacuVnd8m2ueeoaf1uaf1Fzquzzq0pZRSyiMaJEoppTyiQXJu\nf+/sBnQC7XPP0NP63NP6Cx3UZz1HopRSyiN6RKKUUsojGiQuIjJNRPaJyEERefAs9y8SkWwRyRKR\nf4vI0M5oZ3s6V5/dtpslIkZEuvWMlza8xneISJnrNc4SkTs7o53tqS2vsYjcIiK7RWSXiCzt6Da2\ntza8zk+6vcb7ykOUFAAABb5JREFURaSqM9rZntrQ5wQR+UREtovIThGZ3q4NMMb0+C+s1RZzgWTA\nH9gBDD1jmzC372dirdrY6W33Zp9d24UC64EvgLGd3W4vv8Z3AH/r7LZ2cJ8HAduBCNftPp3dbm/3\n+Yzt/wNr9dZOb7uXX+e/A3e7vh8KFLRnG/SIxDIeOGiMyTPGtADLgOvdNzDG1Ljd7IW1BHB3ds4+\nuzwK/AFo6sjGeUFb+/tt0pY+/xB4yhhTCWCM8f4C3951vq/zbcCrHdIy72lLnw0Q5vo+HDjSng3Q\nILHEAYfcbhe7fnYaEfmJiORivbHe20Ft85Zz9llERgPxxpiVHdkwL2nTawzc7Dr0f1NE4jumaV7T\nlj5fDFwsIp+JyBciMq3DWucdbX2dEZFEYCDwcQe0y5va0udfA/NFpBhYjXUk1m40SCxylp995YjD\nGPOUMSYFeAD4lddb5V3f2GcR8QGeBH7eYS3yrra8xiuAJGPMCOAj4GWvt8q72tJnX6zhrSlYn85f\nEJHeXm6XN7Xpb9llDvCmMcbhxfZ0hLb0+Tbgn8aYAcB0YLHrb7xdaJBYigH3T58D+OZDv2XADV5t\nkfedq8+hwHBgnYgUABOA5d34hPs5X2NjzHFjTLPr5vPAmA5qm7e05fe6GHjPGNNqjMkH9mEFS3d1\nPn/Lc+j+w1rQtj7/AHgdwBizEQjEqsPVLjRILJuBQSIyUET8sX7BlrtvICLuf1zXAQc6sH3e8I19\nNsZUG2OijTFJxpgkrJPtM40xWzqnuR5ry2vcz+3mTGBPB7bPG87ZZ+Bd4DsAIhKNNdSV16GtbF9t\n6TMiMhiIADZ2cPu8oS19LgKuAhCRIVhBUtZeDfBtrwfqzowxdhG5B1iLNQPiJWPMLhF5BNhijFkO\n3CMiVwOtQCVwe+e12HNt7PO3Rhv7e6+IzATsQAXWLK5uq419XgtcIyK7AQfwC2PM8c5rtWfO4/f6\nNmCZcU1j6s7a2OefA8+LyH1Yw153tGff9cp2pZRSHtGhLaWUUh7RIFFKKeURDRKllFIe0SBRSinl\nEQ0SpZRSHtEgUcpDIvJrEbm/C7SjwHUtiFIdSoNEKaWURzRIlDoLEeklIqtEZIeI5IjIre6f+EVk\nrIisc9tlpIh8LCIHROSHrm36ich617oXOSJyhevnz4jIFtf6H79xe84CEXlMRDa67k8TkbUikisi\ni1zbTHE95juuNUSePVvNJBGZLyKbXM/9nIjYvPn/pXo2DRKlzm4acMQYM9IYMxxYc47tR2CVzpkI\n/LeI9AfmAmuNMaOAkUCWa9uHjTFjXfuki8gIt8c5ZIyZCGwA/gnMwqpz9ojbNuOxrlROBVKAm9wb\n4iqBcSswyfXcDmDeefRdqfOiJVKUOrts4AkR+T2w0hizQeRsRVZPes8Y0wg0isgnWG/2m4GXRMQP\neNcYcyJIbhGRu7D+/vphLTS003XfiRIe2UCIMaYWqBWRJreqvJuMMXkAIvIqcDnwpltbrsIqOLnZ\n1eYgoLuvM6K6MA0Spc7CGLNfRMZgldx+XEQ+wKrBdeIoPvDMXb76EGa9iEzGOlJZLCJ/xDrSuB8Y\nZ4ypFJF/nvFYJ6oPO92+P3H7xN/rV57rjNsCvGyMeegc3VSqXejQllJn4RqaajDGLAGeANKAAk6V\nlr/5jF2uF5FAEYnCWttjs2vhpFJjzPPAi67HCAPqgWoRiQWuvYDmjXdVevXBGsL69xn3/wuYJSJ9\nXH2JdLVFKa/QIxKlzi4V+KOIOLEqPt+NNUT0ooj8EvjyjO03AauABOBRY8wREbkd+IWItAJ1wEJj\nTL6IbAd2YZVr/+wC2rYR+J2rjeuBd9zvNMbsFpFfAR+4wqYV+AlQeAHPpdQ5afVfpboREZkC3G+M\nmdHZbVHqBB3aUkop5RE9IlFKKeURPSJRSinlEQ0SpZRSHtEgUUop5RENEqWUUh7RIFFKKeURDRKl\nlFIe+f+JOS/i9+8ayAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f931839a9b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize results\n",
    "print(\"Best: %f using %s\" % (gsearch3_1.best_score_, gsearch3_1.best_params_))\n",
    "test_means = gsearch3_1.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = gsearch3_1.cv_results_[ 'std_test_score' ]\n",
    "train_means = gsearch3_1.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = gsearch3_1.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "pd.DataFrame(gsearch3_1.cv_results_).to_csv('my_preds_subsampleh_colsample_bytree_1.csv')\n",
    "\n",
    "# plot results\n",
    "test_scores = np.array(test_means).reshape(len(colsample_bytree), len(subsample))\n",
    "train_scores = np.array(train_means).reshape(len(colsample_bytree), len(subsample))\n",
    "\n",
    "for i, value in enumerate(colsample_bytree):\n",
    "    pyplot.plot(subsample, -test_scores[i], label= 'test_colsample_bytree:'   + str(value))\n",
    "#for i, value in enumerate(min_child_weight):\n",
    "#    pyplot.plot(max_depth, train_scores[i], label= 'train_min_child_weight:'   + str(value))\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'subsample' )                                                                                                      \n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'subsample_vs_colsample_bytree1.png' )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
